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This is in response to the appeal brief filed 12/03/2009 appealing fi-om the Office action 
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(1) Real Party in Interest 

The examiner has no comment on the statement, or lack of statement, identifying by 
name the real party in interest in the brief 

(2) Related Appeals and Interferences 

The examiner is not aware of any related appeals, interferences, or judicial proceedings 
which will directly affect or be directly affected by or have a bearing on the Board's decision in 
the pending appeal. 

(3) Status of Claims 

The statement of the status of claims contained in the brief is correct. 

(4) Status of Amendments After Final 

The examiner has no comment on the appellant's statement of the status of amendments 
after final rejection contained in the brief 

(5) Summary of Claimed Subject Matter 

The examiner has no comment on the siimmary of claimed subject matter contained in 
the brief 

(6) Grounds of Rejection to be Reviewed on Appeal 

The examiner has no comment on the appellant's statement of the grounds of rejection to 
be reviewed on appeal. Every ground of rejection set forth in the Office action from which the 
appeal is taken (as modified by any advisory actions) is being maintained by the examiner except 
for the grounds of rejection (if any) listed under the subheading "WITHDRAWN 
REJECTIONS." New grounds of rejection (if any) are provided under the subheading "NEW 
GROUNDS OF REJECTION." 
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(7) Claims Appendix 

The examiner has no comment on the copy of the appealed claims contained in the 
Appendix to the appellant's brief. 



(8) Evidence Relied Upon 

5280530 Trew et al. 01-1994 

6483538 Hu 11-2002 

2003/0231791 Torre-Bueno et al. 12-2003 

2002/0044682 Weil et al. 04-2002 

6208769 Pankratov, KiriU K. 03-2001 



(9) Grounds of Rejection 

The following grounds of rejection are applicable to the appealed claims, the same prior 
art relied upon in the previous office action has been used. However, further detail of the prior 
art and explanation of the claim language interpretation has been included. 



Claim Rejections - 35 USC § 103 

1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 
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2. Claims 61, 67, 69-72, 90, and 98 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Trew (US 5280530) in further view of Hu (US 6483538). 



Instant claim 61 : A method for determining coordinates of a feature comprising: 
providing a first image including the feature, the first image comprising a plurality of pixels; 
[Trew has taught in column 2 line 20-37, column 3 lines 65+, and column 4 lines 20-65 the use 
of an initial template image including the desired feature.^ 

determining a first estimate of coordinates of the feature to within a fraction {This limitation does 
not preclude the estimate being an integer value, and thus is currently being interpreted as being 
an integer value.] of a pixel; [Trew has further taught, in the sections referred to above, the 

determination of the coordinates of a feature in a series of images, but does not teach a method 
of correlation wherein coordinates are determined with sub-pixel precision. However, Hu has 
taught a method of aligning a selected reference mage block (containing features of the 
reference image) with a region of a target image (see column 2 lines 31-65). Furthermore, Hu 
has taught the correlation of the image block (containing a feature of the reference image) with a 
target image (containing a region with the same features as those of the reference image block) 
to determine the position of the feature contained in the reference image block in the target 
image. In column 2 lines 5-65 the measurement of a coarse (first estimate) and then a fine 
(second estimate) of the position of the reference feature (represented by the reference block) are 
determined using correlation methods. Thus both Hu and Trew have taught the determination of 
the position of an object (feature) in a target image by utilizing a template or reference object 
(reference block ofHu) and a method of searching the target image to locate the feature within 
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that image. It would have been obvious to one of ordinary skill in the art at the time of the 
invention to substitute the feature locating method ofTrew with the correlation coarse to fine 
method ofHu to increase the accuracy of the positional measurement to sub-pixel precision as 
was taught by Hu. Thus Trew in view of Hu has taught the sub-pixel positioning of a feature 
within an image using correlation.] 

translating the feature relative to the pixels [Note: The feature described by Trew in view ofHu 
is both contained in the template or reference block that is described by pixels, and in the target 
image that is being searched for a position of the feature contained in both reference (template) 
and target image data. Furthermore, in the method of Hu the reference block containing the 
feature is shifted in relation to pixels of the target image after the coarse positional estimate has 
been performed (first estimate of coordinates).] by a pixel translation value, wherein the sum of 
the pixel fraction \As discussed above, the value of the pixel fraction is an integer value (since an 
integer value can be written in fractional form).] and pixel translation value is an integer value 
[As per the above and below discussion the coarse (first estimate) of the pixel location of the 
feature is an integer value, and the pixel translation value (integer shift value taught by Hu) is an 
integer value. Mathematically, the sum of an integer (such as the pixel translation value) and an 
integer (the first estimate or pixel fraction) is always an integer value as required by the 
language of the claim.]; [Hu teaches in lines 5-20 of column 3 the shift of the measured pixel 
position to the nearest integer pixel position using the "nearest integer pixel position shift". 
Thus the shift referred to by Hu is the total shift value that translates the measured fractional 
pixel value to a "nearest integer pixel". Thus it has been established thatHu teaches shifting of 
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the feature (contained in the reference block ofHu) relative to the pixel values (of the target 
image or first image) to the "nearest integer".^ 

determining a second estimate [Referred to as fractional or fine estimate.] of coordinates of the 
translated feature to within a fraction of a pixel; and [Hu has taught in column 3 lines 21-44 the 
determination of an additional fractional estimate (fine estimate) of coordinates using the 
translated image.] 

summing the pixel fractions of the first estimate with the second estimate to derive a refined 
estimate of coordinates. [See Hu lines 30-32 of and 38-40 of column 3 (XA+J^= first estimate + 

second estimate). Therefore, Trew in view of Hu has taught the defining of a feature within a 
template image (Trew defines a template image containing pixels representing a desired 
feature), the coarse estimate of the position of the feature in the first image (target image) using 
the template feature (the feature contained in both the template and target image is the same 
feature), the translation of the template and the feature contained within relative to the pixels of 
the target image by the amount determined in the coarse estimate, the fine estimate (fractional) 
of the position of the feature in the first image (target image) using the template feature, and the 
summing of the first estimate and second estimate values.] 

Instant claim 90: A method for determining coordinates of an object, the method comprising the 
steps of: 

capturing at least one first image and at least one second image of the object, each image 
being captured having different coordinates with respect to the other; [See the discussion of 
claim 61, wherein the capturing of a video sequence and the correlation of an image with a 
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feature to at least one other image containing said feature by Trew in view of Hu. Furthermore, 
the teachings of either Trew or Hu do not specifically teach or require that each image has the 
same coordinates. However, Examiner takes official notice that one of ordinary skill in the art 
at the time of the invention would have known that requiring each pixel of each image in a 
sequence of images to have the same coordinates would have required additional steps and 
devices not taught or required by the cited prior art. Therefore, in view of the knowledge of one 
of ordinary skill in the art at the time of the invention, the video images of Trew in view ofHu 
would have different coordinates with respect to the other. ^ 

determining the position of the object within each image, wherein determining includes; 
[See the discussion of claim 61.] 

providing the first image including a feature, the first image comprising a plurality of 
pixels; [See the discussion of claim 61.] 

determining a first estimate of coordinates of the feature to within a fi-action of a pixel; 
See the discussion of claim 61.] 

translating the feature relative to the pixels by a pixel translation value, wherein the sum 
of the pixel fraction and pixel translation value is an integer value; [See the discussion of claim 
61.] 

determining a second estimate of coordinates of the translated feature to within a fraction 
of a pixel; [See the discussion of claim 61.] 

summing the pixel fractions of the first estimate with the second estimate to derive a 
refined estimate of coordinates; and [See the discussion of claim 61.] 
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comparing the determined positions of the object to determine dimensional changes. 
Trew teaches the measurement of the displacement of the features in column 2 lines 30-40.] 

Instant claim 98: The method of claim 61, wherein the first image is captured with an image 

capture device. [As per the discussion of claim 61, both Hit (video test image) and Trew have 
taught the capturing of the first image with an image capture device.] 

Instant claim 67: The method according to claim 61, wherein the translating step, second 
determining step and summing step are repeated at least once. [Hu in column 3 lines 20-45 
iterates the algorithm a specified number of iterations and checks the measured value with a 
noise value.] 

Instant claim 69: An apparatus for determining a position of an object comprising: 

an image capture device arranged to provide a captured image encompassing the object, 
the captured image comprising a pliirality of pixels; and [Trew in lines 45-55 of column 5 
describes a camera for capturing the images containing the feature to be tracked.] 

an image processor arranged to receive the captured image and determine the position of 
the object by executing the method of claim 61 . [Trew in lines 45-55 further describes a means 
for performing the processing, but does disclose the specific means. However, Hu in figure I 
and column 2 lines 30-50 discloses a video processor for performing the video processing 
method.] 
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Instant claim 70: The apparatus according to claim 69 further comprising: 

a monitor arranged to receive and display the captured image; and [Trew teaches a 
display means for displaying the images in column 6 lines 25-35.] 

an object selection means arranged to select a further object within the displayed image 
and to identify the further object to the image processor. [Trew teaches in column 6 lines 55 the 
selection (tracing of feature to be used as template) of the template by automatic or manual 
means. Furthermore, Tracing as taught by Trew must be done on a displayed image, and thus 
Trew teaches a selection means wherein the object is selected within the displayed image.] 

Instant claim 71 : An apparatus for determining a position of an object comprising: 

an image capture device arranged to sequentially provide a plurality of captured images 
of an object, each captured image comprising a plurality of pixels; and {Trew in lines 45-55 of 
column 5 describes a camera for capturing a series of images containing the feature to be 
tracked.] 

an image processor arranged to sequentially receive the plurality of captured images and 
determine the position of the object from the plurality of captured images by executing the 
method of claim 61; and \Trew in lines 45-55 further describes a means for performing the 
processing, but does disclose the specific means. However, Hu in figure 1 and column 2 lines 
30-50 discloses a video processor for performing the video processing method.] 

a position comparator arranged to compare the determined position of the object for the 
plurality of captured images and identify whether the determined position changes in the 
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plurality of captured images. [Trew teaches the measurement of the displacement of the features 
in column 2 lines 30-40.] 

Instant claim 72: The apparatus according to claim 71 further arranged to determine the change 

in the determined position, the change selected from the group consisting of magnitude, 
direction, and combinations thereof. [Trew teaches the measurement of the displacement 
(displacement is magnitude of positional change, but also includes directional information for 
tracking) of the features in column 2 lines 30-40.] 

3. Claims 91 and 99 are rejected under 35 U.S.C. 103(a) as being unpatentable over Trew in 
view of Hu as applied to claim 61 above, and fiirther in view of Torre-Bueno et. al. (US 
2003/0231791). 2002/0044682 

Instant claim 91 : The method of claim 61, wherein the refined estimate of coordinates is 
recorded on a computer readable medium. [Trew in view ofHu have taught the detection of a 
feature using correlation, and the measurement of the shift and position in the image, but have 
not taught the store of the positional information. However, Torre-Bueno has taught in 

paragraph 0070 the measurement of an objects location in an image and the storage (on a 
computer readable medium) of the information for later using in processing the information. It 
would have been obvious to one of ordinary skill in the art to modify the teachings of Trew in 
view ofHu (object detection and subpixel positional measurement) with the teachings of Torre- 
Bueno to store the information for later use.] 
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Instant claim 99: The method of claim 90, wherein the coordinates are recorded on a computer 
readable medium. [See the discussion of claim 91.] 

4. Claims 92-97 are rejected under 35 U.S.C. 103(a) as being unpatentable over Trew in 
view of Hu as applied to claim 61 and 90 above, and further in view of Well et. al. (US 
2002/0044682). 

Instant claim 92: The method of claim 90, further comprising determining a 2-dimensional 
position of the feature within the at least first image and the at least second image, wherein a 
position of the at least second image is known relative to the at least first image. [Trew in view 
ofHu has taught the positional determination of an object from one image in the other image 
(second image), hut has not taught that the position of the at least second image is known related 
to the first image. However, Well has taught the positioning of stereoscopic images (necessary 
to know relative position in order to determine 3-D position of object captured by both imagers), 
and the correlation of a feature from one image to the other in order to determine the positioning 
of the object capture by the imagers (paragraph 0086). It would have been obvious to one of 
ordinary skill in the art to modify the positional determination method described by Well using 
the subpixel method taught by Trew in view of Hu to increase the accuracy of the positional 
information (3-D position includes 2-D position information).] 
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Instant claim 93: The method of claim 92, fUrther comprising calculating a 3 -dimensional 
position of the feature from the 2-dimensional position for the at least two images. [See the 
discussion of claim 92.] 

Instant claim 94: The method of claim 61, further comprising determining coordinates of the 
feature within a second image, the position of the second image being known relative to the first 
image. [See the discussion of claim 92.] 

Instant claim 95: The method of claim 61 , further comprising determining a difference in 
position of the feature between the first image and at least one second image, wherein the at least 
one second image includes coordinates and has a position known relative to the first image. [See 
the discussion of claim 92, wherein the difference between the feature positions is an inherent 
part of determining the 3-D position of the object due to parallax between the stereoscopic 
imaging devices.] 

Instant claim 96: The method of claim 61, fiirther comprising superimposing the first image and 

a second image to provide a superimposed image, wherein the position of the second image is 
known relative to the first image, and wherein the feature is substantially in registration. 
[Regarding the relative positioning, see the discussion of claim 92. Additionally, the method of 
correlation taught by Trew in view of Hu inherently includes the overlaying (superimposing) of 
an image with the feature over the other image and iteratively calculating the correlation value 
shifting until the feature is in registration (highest correlation value).] 
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Instant claim 97: The method of claim 61, wherein the method is applied for monitoring an 
aircraft structure. [As per the discussion of claims 92-96, the use of a series of images to monitor 
and determine the position of an object has been taught, but the above cited prior art references 
do not teach the monitoring of an aircraft structure. However, one of ordinary skill in the art at 
the time of the invention would have expected the disclosed system to be capable of detecting 
targeted structures, objects, or features in a plurality of environments and situations not 
explicitly listed by the cited prior art that require the tracking or positional determination of an 
object using a plurality of imagers that have been taught by the above cited art.] 

5. Claims 62-64, 66, and 77 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Trew in view of Hu as applied to claim 61 above, and fiirther in view of Pankratov (US 
6208769). 

Instant claim 62: The method according to claim 61, wherein each of the first and second 
determining steps comprise: 

correlating the feature and the image using a predetermined correlation fiinction to 
determine coordinates of the feature to the nearest pixel; [As per the rejection of claim 61 Trew 
in view of Hu has taught the use of a correlation function to determine the coordinates of a 
feature with sub-pixel accuracy.] 

evaluating the correlation fimction at a plurality of pixel positions in the neighborhood of 
the determined coordinates to provide a plurality of values; [Hu teaches in lines 35-45 of column 
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3 the further correlation of the image to determine the fractional pixel shift, but does not limit the 
correlation to pixels in the neighborhood of the determined coordinates. However, in 
calculating the correlation values of the image as a whole Hu also teaches the evaluation of 
correlation values within the neighborhood of the coordinates since they are included within the 
image being evaluated by the correlation function.'] 

fitting the plurality of values to a fiirther function; and [Hu does not teach fitting the 
result (often referred to as score) of the correlation function to an equation or surface. 
However, as is evidenced by the teachings of Pankratov the fitting of correlation functions to 
surfaces for the purpose of subpixel measurement was known to one of ordinary skill in the art 
(see column 3 lines 32-52 of Pankratov). It would have been obvious to one of ordinary skill in 
the art at the time of the invention to modify the correlation method ofHu to more accurately 
determine the subpixel positioning using the fitting and determining method taught by 
Pankratov.] 

differentiating the further function to determine its turning point, whereby coordinates 
corresponding to the turning point provide coordinates of the feature. [Pankratov teaches the 
determination of the maximums (turning point) by differentiating the function and determining 
the points that make the functions equal to zero (see column 3 lines 42-45 of Pankratov).] 

Instant claim 63: The method according to claim 62, wherein the correlation fiinction is 
evaluated at a plurality of sub-pixel positions. [Pankratov teaches the evaluation of the function 
at regular intervals (column 5 lines 8-12 and figure 6 -points J, I, G, and H).] 
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Instant claim 64: The method according to claim 63, wherein the sub-pixel positions are closer 
in proximity to the determined coordinates than the pixel positions. [See figure 6 (Pankratov) 
points J, I, G, and Hare closer to O (determined coordinates) than the pixel positions (points A, 
M,E,andD).] 

Instant claim 66: The method according to claims 62, wherein the predetermined correlation 
function is a normalized greyscale correlation function. [Pankratov teaches normalized 
correlation in lines 65+ of column 3. The normalized correlation is a shorthand reference to 
normalized greyscale correlation.] 

Instant claim 77: A method for determining coordinates of a feature comprising: 

providing at least one image including the feature, the at least one image comprising a 

plurality of pixels; [See the rejection of claim 61.'] 

correlating the feature and the at least one image using a predetermined correlation 

function to determine coordinates of the feature to the nearest pixel; [See the rejection of claim 

62.] 

evaluating the correlation function at a plurality of sub-pixel positions in the 
neighborhood of the determined coordinates to provide a plurality of values and fitting the 
plurality of values to a further function; and [See the rejection of claims 62-63.] 
differentiating the further function to determine its maximum, whereby coordinates 
corresponding to the maximum are coordinates of the feature to within a fraction of a pixel. [See 
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the rejection of claim 62 wherein the maxima 
zero (Pankratov).]. 
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at points when the differentiated function is 



(10) Response to Argument 
Per claim 61: 

Issue 1: Appellant asserts (Brief, p. 5) that neither Trew nor Hu determines a first 
estimate of coordinates of the feature to within a fraction of a pixel. Additionally, 
Appellant has argued that Hu expressly states that "for the first iteration, only the nearest 
integer pixel shift position is used". Examiner agrees that Hu has taught an integer shift, 
but disagrees with the argument that Hu does not determine a first estimate of coordinates 
of the feature to within a fraction of a pixel. The Appellant's currently presented claim 
language "determining a first estimate of coordinates of the feature to within a fraction of 
a pixel" has only required that the measurement of the feature of the image be within a 
fraction of a pixel, but does not fiuther require that fractional value be a non-integer 
value. Since an integer value can be written in fractional form, the first estimate referred 
to in the language of the claim can be an integer value (as admitted by Applicant in the 
interview dated 09/13/2010). Therefore, the first estimate (integer value) measurement 
value of the claim and the integer valued shift corresponding to the integer value 
positional measurement taught by Hu in column 3 lines 5-20 have met the limitations of 
the currently presented claim language. 
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Issue 2: Appellant asserts (Brief, p. 6) that neither Trew nor Hu sum the pixel fractions 
of the first estimate with the second estimate to derive a refined estimate of coordinates. 
Appellant has further argued that Hu has not taught the first estimate being a pixel 
fraction. Examiner agrees that Hu has not taught the summing of two fractional estimates, 
but the claim (see the above discussion of Issue 1 with regards to the fractional values 
corresponding to integer values written in fractional form) only requires two values, 
corresponding to the first and second estimate (as per the discussion of Issue 1 the values 
could be fractional, integer, or both), be summed. Hu has taught in column 3 lines 5-20 
the summation of the second fractional estimate (denoted as Xf by Hu) with a first 
estimate (denoted as Xdelta by Hu). Therefore, Trew in view of Hu has taught the 
currently claimed limitations. 

Issue 3: Appellant asserts (Brief, p. 7) that neither Trew nor Hu translate the feature 
relative to the pixels by a pixel translational value, wherein the sum of the pixel fraction 
and pixel translation value is an integer value. As per the rejection of claim 61 and the 
discussion of Issue 1, the pixel fraction value of the claim is an integer value 
corresponding to the initial measurement taught by Hu, and the pixel translational value 
is taught by Hu as the nearest integer shift value, which represents the amount of shift to 
be carried out in the image directly proportional to the initial measurement value (thus is 
also an integer value). Furthermore, as was well known to one of ordinary skill in the art 
at the time of the invention, the sum of two integer values will always be an integer 
value, thus the sum of the initial measurement of Hu and the shift value of Hu will always 
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be an integer value. Appellant further asserts on pages 8-10 that the feature is not 
translated relative to the pixels. The Examiner understands that the feature, as intended 
by Appellants in the written disclosure of the application, moves relative to the pixels in a 
manner different than interpreted by the Examiner in the currently presented claims and 
as has been described by the currently cited prior art. However, both Trew and Hu have 
taught the shifting of a feature contained in a template (Trew has taught the template, Hu 
has taught a block template containing pixels that inherently represent at least some 
feature of the image) over a target image (having pixels) containing the same feature as 
represented in the template during the stage of the method (as taught by Hu) that requires 
the shift (by an integer value) of the template over the target image after the coarse 
(initial) estimate has been made. Thus, Trew in view of Hu has taught that the feature 
exists both within the template and the image being processed to locate said feature of the 
template within said image. Furthermore, Trew in view of Hu have taught the shifting 
(spatial translation) of the template containing the feature over the image containing 
pixels, based on the measurements obtained regarding the determined position of the 
feature represented by the template within the image being processed. This shifting 
(spatial translation) of the template containing the feature over the image being 
processed, as taught by the prior art, is being interpreted as the "translation of the feature 
relative to the pixels" as has been required by the currently presented claim language. 
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(11) Related Proceeding(s) Appendix 

No decision rendered by a court or the Board is identified by the examiner in the Related 
Appeals and Interferences section of this examiner's answer. 

For the above reasons, it is believed that the rejections should be sustained. 

Respectfully submitted, 

/NATHAN BLOOM/ 
Examiner, Art Unit 2624 

Conferees: 
/Vu Le/ 

Supervisory Patent Examiner, Art Unit 2624 
/Bhavesh Mehta/ 

Supervisory Patent Examiner, Art Unit 2624 



